Designing experiments informed by observational studies

Abstract The increasing availability of passively observed data has yielded a growing interest in “data fusion” methods, which involve merging from observational and experimental sources to draw causal conclusions. Such methods often require precarious tradeoff between the unknown bias dataset often-large variance dataset. We propose an alternative approach, avoids this tradeoff: rather than using for inference, we use it design more efficient experiment. consider case stratified experiment with binary outcome suppose pilot estimates stratum potential variances can be obtained study. extend existing results generate confidence sets these variances, while accounting possibility unmeasured confounding. Then, pose problem as regret minimization subject constraints imposed by our sets. show that converted into concave maximization solved conventional methods. Finally, demonstrate practical utility Women’s Health Initiative.